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Abstract 

The purposes of this paper are to explain the development processes of an intensive summer program for 
English language learners and to discuss course improvement strategies based on the evaluation 
outcomes. This 60-hr partnership program between local schools and a university was developed to 
improve the language proficiency of increasing numbers of international students in the southeastern 
United States. Following a conventional instructional design model, the authors conducted needs 
assessments with parents and teachers and developed and implemented a one-month intensive English 
program emphasizing the improvement of reading and writing skills and test preparation. Formative 
and summative evaluations were conducted to identify areas of improvement and to examine the 
effectiveness of the program. This paper reports the evaluation results and discusses three areas that 
need to be considered to develop a quality summer English program: (a) the incorporation of 
standardized tests, (b) the creation of comfortable learning environments, and (c) the development of 
curriculum content. 


Introduction 

Many scholars claim that the Information Age of the 21st century demands a new 
instructional approach that focuses on the diverse needs of students (Halverson, Grigg, 
Prichett, & Thomas, 2007; Toffler & Toffler, 2006). Reighluth (1997) particularly emphasizes the 
importance of customized instruction. He asserts that as people acclimate to customized 
marketing and communications (e.g., cell phones and Tivo), they are likely to expect such 
customization in all spheres of life, including education. Thus, it is critical for program 
designers to examine the needs of students and their parents and to develop strategies not 
only to meet their specific demands but also to ensure the quality of the program (Brisk, 1998). 

The purposes of this paper are to describe the development process of an intensive summer 
program for English language learners (ELLs) and to discuss program improvement strategies 
based on the evaluation outcomes. With the influx of foreign residents, the need for academic 
support during the summer break has increased in the southeastern United States. To support 
these new residents, we (two faculty members at a local university) conducted an extensive 
needs analysis and developed and implemented an English language summer program over 
the course of two years. The 60-hour program was a partnership project between two local 
districts and the university and was open to all international residents. In this paper, we first 
provide background information about developing such a program and then describe the 
outcomes of the initial project. Based on first-year experiences, we redesigned the program 
according to an instructional systems design model. A detailed explanation of that model and 
the program development process follows. We conclude the paper by suggesting areas that 
need consideration in order to develop a quality summer program for ELLs. 
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Background of the Study 

With the opening of many automobile manufacturing companies in the local community, 
the international population has dramatically increased, including an especially high number 
of Korean families. Previous studies have indicated that family members of internationals 
undergo hardships while acclimating to a foreign environment (Ali, Van der Zee, & Sanders, 
2003; Copeland & Norell, 2002; Ward & Kennedy, 1994). When there is inadequate support for 
these newcomers, children and their family members are at risk of experiencing emotional, 
psychological, and social difficulties. Studies also indicate significant academic achievement 
gaps among ELLs. For example, Echevarria, Short, and Powers (2006) report that 11% of ELLs 
in grade 7 who took the California reading test in 2002 scored at or above the 50th percentile, 
compared to 57% of minority students who were proficient in English and 48% of all students 
who took the test. Further, Klingner, Artiles, and Barletta (2006) argue that ELLs tend to 
exhibit lower overall academic achievement than their peers, particularly in literacy. ELLs 
have higher dropout rates and are more frequently placed in lower ability groups than native 
speakers (Echevarria et al., 2006). Culturally, Korean parents have high expectations for their 
children's education and want to provide additional academic support outside of school to 
help their children excel (Zhou & Kim, 2006). Thus, a Korean family typically spends 15% to 
30% of its budget on private education to provide additional learning opportunities after 
school or during summer and winter vacations (Lartigue, 2000). 

Based on these studies, we wanted to see whether academic or social support programs 
were necessary for newly arrived Korean residents. Lienee, a needs assessment was 
administered through Korean churches, personal contacts, and the Korean Student Association 
Web site in January of 2008. A total of 29 parents completed the needs assessment. These 29 
parents, each of whom expressed a desire for their children to be enrolled in a summer English 
program, represented a total of 51 students. Of the 29 parents, 9 expressed a desire to attend 
English sessions themselves. These results showed that the students' primary needs were in 
the subject areas of writing, grammar, and reading. Greater need was found in writing and 
reading for lower grades (K-3) and in writing and grammar for grades 4-7, with writing being 
the first priority across all the grade levels. The parents of high school students also indicated 
interest in a Scholastic Aptitude Test (SAT) preparation course. The parents' needs focused on 
conversational English, followed by writing and grammar. 

Based on these results, we developed a six-week summer English program with four 
different classes: Reading and Writing, English for Speakers of Other Languages (ESOL), 
Scholastic Aptitude Test (SAT) Preparation, and Adult English. The program took place from 
June 2 to July 17, 2008, for 3 hours per day from Monday to Thursday at a local school. In total, 
102 people (81 children and 21 adults) registered for the program. The program was funded by 
student tuitions and donations from local city offices, the university, and community 
corporations. Practicing teachers in local school districts and graduate students majoring in 
education or English at the university were recruited to teach in the program. 

To investigate the effectiveness of the program, the students' English language skills and 
self-concepts were measured before and after the classes. Their English skills were measured 
through the use of the Basic Achievement Skills Inventory (for students in the regular reading 
and writing classes), BEST Literacy (for students in the adult class), and teacher-designed pre- 
and posttests (for students in the ESOL and SAT classes). The posttests revealed that the 
overall means in all areas increased except for the total reading scores in the reading and 
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writing class. The participants' self-concepts were measured by the Piers-Harris Children's 
Self-Concept Scale, Second Edition (for grades 3-11), and the Quality of Life Inventory (for 
adults) before and after the program. The results indicated that the overall means of self- 
concept of all students increased except for the students in the SAT class. However, the 
differences were small for all participants, and no statistically significant impact on self- 
concept was found when it was measured using the two instruments. 

Although the students' test scores demonstrated the effectiveness of the program, the 
implementation experiences and informal discussions with parents afterward indicated areas 
for improvement. Moreover, as we observed the benefits of the program, we wanted to extend 
the program to all of the international residents in the community. Thus, we decided to 
redesign it following a systemic design model to create a higher quality educational product. 
In the next section, a conventional instructional design model, Analysis-Design-Develop- 
Implement-Evaluation (ADDIE), is explained, followed by the details of the program redesign 
process. 


Instructional Systems Design Model: ADDIE 

ADDIE is an instructional systems design model that has been used to develop 
instructional materials, educational programs, and corporate and military trainings (Molenda, 
2003). This model provides program designers general guidelines on critical components and 
their sequences (Gordon & Zemke, 2000). It has five steps: analysis, design, development, 
implementation, and evaluation; the outcome of each step informs the subsequent step. Major 
activities in each phase are summarized below. 

• Analysis: In this stage, designers examine specific learner needs, learning 
environments, and curricula. Learners' prior knowledge, general learning styles, and 
specific content areas are explored. Additionally, parents' and stakeholders' 
perspectives must be understood to establish the need for a program and to investigate 
existing support structures. Analysis methods may include surveys, interviews, and 
observations of students and learning environments (Rowland, 1993). 

• Design: Specifying learning objectives, exploring assessment strategies, and selecting 
teaching tactics occur in the design phase. Molenda, Pershing, and Reighluth (1996) 
claim that the purpose of the last stage of analysis and the first stage of design is to 
define clear instructional and learning goals. Well-designed objectives should include 
(a) target learners, (b) expected performances, (c) the conditions under which the 
identified activities exist, and (d) the quantitative and qualitative criteria by which the 
learner performances were evaluated (Mager, 1997). Based on the specified learning 
objectives, designers develop test items that measure learner performances. Types of 
assessments may include written tests (e.g., multiple choice, short answer, essay), oral 
presentations, role-plays, or portfolios. Finally, overall learning strategies (e.g., case 
studies or project-based learning) and specific learning tactics (e.g., teaching methods, 
media, or instructional procedures) are selected. 

• Development: Actual learning materials are developed in this stage. More specifically, 
learning activities, program materials (e.g., teacher manuals and student worksheets), 
and assessment tools are developed. All prototypes are examined by conducting one-to- 
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one tryouts, small-group tryouts, or expert reviews, so any errors can be fixed in the 
early stages of development (Molenda et al., 1996). 

• Implementation: During this phase, the developed materials are put into action. 
Teachers who will deliver the instruction should be well aware of the program goals, 
program needs, and specific teaching strategies. If not, a workshop or training needs to 
be provided before instruction takes place in the classroom setting. 

• Evaluation: This is a systematic process of determining the quality and effectiveness of 
the program. Although it is considered the last stage of the ADDIE model, evaluation is 
an ongoing activity during the entire process. Scriven (1996) distinguishes between 
formative and summative evaluation: Formative evaluation is a process of identifying 
performance aspects that need to be improved and offering constructive advice, while 
summative evaluation is the process of making judgments about program adoption, 
continuation, or expansion based on the examination of the achievements of program 
goals. Both types of evaluation are important in this design model. 

One critical characteristic of the ADDIE model is that the procedures are not always linear. 
Although the outcome of one stage becomes the input for the next stage in many cases, each 
process develops as necessary. For example, analysis of student needs or development of new 
materials may reoccur at later stages to ensure the quality of the program. Most importantly, 
ADDIE is a guideline, not a formula (Gordon & Zemke, 2000; Molenda, 2003). Depending on 
the situation, program designers can add or delete tasks in each stage. Conducting formative 
evaluation is important when examining unique situations and developing program 
improvement plans (Fraenkel & Wallen, 2003). By collecting information about the 
effectiveness of prototypes and uncovering barriers to success, program developers can 
continue to make improvements by integrating new design strategies. Following this ADDIE 
model, we conducted a new needs assessment and redesigned the summer program. 

The Redesign of the Summer English Program 
Analysis: Identifying New Needs 

A new needs assessment was conducted to examine areas for improvement based on the 
first-year trial. To this end, we interviewed seven parents, as well as the staff member who 
managed the program in 2008. The findings indicated three areas for improvement: (a) 
instruction, (b) communication with parents, and (c) student placement. 

All parents expressed the desire for their children to have more strict and demanding 
instruction. They wanted teachers to assign homework every day and give more tests. One 
participant said, "I wish the teacher would have provided more reading and writing 
assignments. My child did not review lessons at home because no homework was assigned. I 
did not like it." Parents often shared specific homework examples that they wanted teachers to 
assign. The participant continued, "For example, if students learn 10 new vocabulary words, 
the teacher could ask students to create 10 sentences with the new words as homework. Or the 
teacher could select several different books in advance and ask students to read one book each 
week. Then, students could read four to six different books while they attended the program." 
Another participant made a similar suggestion: "I had hoped the teacher would ask students 
to memorize lists of vocabularies every day and test them." 
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Parents also felt communication was lacking. They did not know what students were 
learning in the classroom, so they could not check the students' progress. They suggested 
providing a weekly plan informing parents of daily lessons and assignments. One mother said: 

My child liked going to the school, so I was happy about that. However, I was not 100% 
satisfied with the program because what my kid was learning was unclear to me. Moms 
want to see "tangible outcomes." I wanted the program to provide clear objectives in the 
beginning and guide students to achieve those goals with homework and tests... I was 
willing to go over lessons with my child every day, but because I did not know what he 
was learning in class, I could not help him as much as I would have liked. 

Another interviewee suggested providing a program orientation informing parents of the 
curriculum and textbook information. She also wanted to get advice on how parents could 
support their children's education at home. 

The staff member discussed the issue of student placement, given the different levels of 
students. She said many parents complained about their children being placed in 
inappropriate class levels. Some parents wanted to transfer their child from an ESOL class to a 
regular reading and writing class or vice versa. Thus, students were moving in and out of 
classes for more than a week, and parents were dissatisfied with the distractions. The ability 
levels of students in some classes, such as the SAT class, were so varied that both high- and 
low-level students complained. The parents interviewed observed the same issues. One 
participant said: 

My child was in a reading and writing class for fourth graders. In the beginning of the 
program, there were only six students in his classroom, but a new student joined almost 
every day and the class size soon doubled.... I know some parents did not want their child 
to be in an ESOL class and asked you to change the class to a regular reading and writing 
class, but I do not think allowing students to change a class whenever parents request it is a 
good idea. I think you need some ways to assign students to the appropriate level. 

After interviewing the parents, we distributed an open-ended survey to teachers in local 
schools to get advice on curriculum development. The questions included (a) areas in which 
international students were particularly weak, (b) suggestions for specific books or content, (c) 
observed behavioral or academic issues among international students, and (d) additional 
advice on program development. Sixteen teachers responded and shared ideas such as specific 
focus content (e.g., U.S. sports, colloquial language and slang, writing a research paper, 
government structures), specific teaching methods (e.g., plays, puppet shows), and observed 
behavioral issues (e.g., shyness, pretending not to understand English). We also organized 
advisory board meetings with assistant superintendents in two local districts, an ESOL 
program coordinator, ESOL teachers, an associate dean in the college, and five people from 
community corporations in order to seek out suggestions on program development and 
implementation. Combining all provided perspectives, we identified the following design 
components: 
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1. Instructional strategies that help students improve reading and writing skills should be 
integrated. 

2. Methods that can assist different levels of students should be included. 

3. Tools to help communicate with parents and students should be provided. 

Design and Development: Searching for Solutions 

The curriculum was developed based on the state language arts curriculum standards and 
English language proficiency standards (i.e., World-Class Instructional Design and 
Assessment). To address the above three design components appropriately, we reviewed 
similar programs, consulted with local ESOL teachers, and developed three program 
implementation ideas. 

Idea 1: Improving English skills through hook reading and keypal exchange. To motivate 
students to improve their reading and writing skills, two activities were developed: book 
reading and writing, and keypal exchange. In the book reading and writing activity, students 
were encouraged to read recommended books and write a one-page reflection paper 
summarizing the story and discussing alternative endings. The number of pages assigned to 
be read before writing a reflection paper was based on grade level. For example, students in 
grade 3 read at least 30 pages, while students in grade 8 read 50. 

The keypal (keyboard pen pal) exchange was a computer-based pen pal activity in which 
students in the summer program and language arts teachers in an ESOL graduate course 
corresponded via e-mail. This activity was designed to help students practice their 
communication skills with native English speakers. Students were required to send at least 
two e-mails per week, discussing issues such as Korean culture and school experiences. 

To encourage more students to participate in these two activities, we created a reward 
system. Students who wrote a reflection paper received a sticker, and students who collected 
more than 12 stickers received awards at the end of the program. For the keypal exchange, 
participant teachers nominated students whose communication skills had significantly 
improved over the course of the activity. 

Idea 2: Helping different levels of students with volunteer teachers. To provide individual 
support to students at lower English skill levels, we recruited volunteers from the university 
and local high schools and assigned them to the preferred age groups. They were asked to 
help individual students who were behind in classes and assist with group work. 
Additionally, the volunteers reviewed the students' book reflection papers and corrected 
grammatical errors, providing verbal and written feedback to individual students. 

Idea 3: Promoting communication using blogs and the program Web site. To promote 
communication among teachers, parents, and students, all teachers were required to develop a 
class blog and post a summary of class activities and homework assignments. The blog was 
intended to help inform parents about daily instruction so that they could better assist their 
children in completing assignments and preparing for activities and quizzes. We also created a 
program Web site with the program's mission, goals, and registration information. On the 
Web site, students and parents could access the recommended book lists and download 
reflection paper templates. 
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Implementation: Meeting the Needs 

The 2009 program took place from June 1 to June 26 at a local elementary school. The 
program offered eight different classes: writing and reading for K-2, 3-4, 5-6, 7-8, and 9-10; 
an adult class; preparation for the Test of English as a Foreign Language (TOEFL); and 
preparation for the Scholastic Aptitude Test (SAT). Classes met for 3 hours per day, five days 
per week, for four weeks. A total of 100 students (92 K-12 students and 8 adults) registered. 
Among them, 98 students were Korean and two were Chinese. 

Originally, we decided to assign students to classes based on their level of English 
language proficiency, and we planned to provide ESOL and regular reading and writing 
classes to groups of two combined grade levels (i.e., ESOL for students in grades 3 and 4, and 
reading and writing for students in grades 3 and 4). Unfortunately, because of state budget 
issues, we could not secure enough classrooms and could only offer eight different classes. 
Therefore, students were assigned to classes based on grade levels, regardless of English 
proficiency. 

The students had lived in the United States for a varied number of years, leading to wide 
differences in their levels of English proficiency. Table 1 shows the number of students and the 
number of years living in the United States. For example, in the seventh and eighth grade 
classroom, six students had lived in the United States for less than six months, and three had 
been here for less than a year. Another three students had lived here for one to three years, and 
the remaining three students had lived here for more than three years. In a clear 
demonstration of the varied experiences, two students had just come to the United States two 
weeks before the course began, while one student had lived here for over ten years. 


Table 1 

Number of Students in Each Class by Years of Living in the United States 


Class 

Less than 

6 months 

6 months to 

1 year 

1 to 3 years 

More than 

3 years 

Total 

K-2 

4 

1 

2 

0 

7 

3-4 Grades 

3 

0 

6 

1 

10 

5-6 Grades 

4 

2 

8 

1 

15 

7-8 Grades 

6 

3 

3 

3 

15 

9-10 Grades 

3 

3 

1 

0 

7 

TOFEL 

3 

0 

8 

4 

15 

SAT 

2 

2 

7 

12 

23 

Adults 

6 

0 

2 

0 

8 

Total 

31 

11 

37 

21 

100 


The eight teachers either had English (or English education) undergraduate degrees or 
were local English teachers. We also had a math teacher who facilitated the SAT math section. 
To provide individual support and to help teachers manage the different levels of students, we 
also recruited volunteer teachers from the university and local high schools. A total of 28 
college and high school students (20 from the College of Education and 8 from high schools) 
assisted students enrolled in the program with individual and group work. Various texts used 
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for different classes including Spectrum Writing, Master Skills English, and Glencoe Backpack 
Reader. 

To motivate students to participate, we provided several awards: (a) highest achievement: 
the student who received the highest scores for the posttest; (b) greatest improvement: the 
student whose test scores demonstrated the most improvement between pre- and posttests; 
and (c) the hardest worker: the student who had worked the hardest, really helped others, or 
really stood out. For each award, one student was selected in each class. Additionally, awards 
were given to (d) the most prolific readers and writers: students who submitted 12 or more 
reflection papers, and (e) the most active Keypalers: students whose writing skills had 
improved significantly while participating in the Keypal activity. A total of 62 students 
received awards on graduation day. Each winner received a $20 gift card to a bookstore 
donated by community corporations. 

Evaluation: Improving the Program 

To continue to identify new needs and examine the effectiveness of the program, formative 
and summative evaluations were conducted throughout the implementation process. Based on 
the formative evaluation results, course structures were slightly modified in the middle of the 
program. The summative evaluation results suggested strengths and weaknesses. The 
following four questions guided the evaluation processes: 

1. To what extent did the program influence students' English proficiency? 

2. To what extent did the program influence students' self-concept? 

3. How did the participants (students, parents, volunteers, and teachers) perceive the 
program? 

4. What activities could be further integrated to create a more satisfying program for 
participants? 

Formative evaluation. Formative evaluation data were collected to ensure that the 
activities were implemented as planned and to obtain feedback from students, parents, and 
teachers. Data were gathered via ongoing informal interviews, a midterm survey, and 
classroom observations. 

Informal interviews were conducted with students, teachers, volunteers, and parents in a 
corridor of the school. The purpose of these interviews was to monitor students' and parents' 
concerns and explore teachers' and volunteers' needs. Ongoing informal interviews helped us 
identify needs that required immediate attention in order to make the program more satisfying 
to students and parents. For example, many parents and students in the third and fourth grade 
class indicated disappointment in the level of the textbook during the first week of the 
program. Although only three students needed ESOL lessons, the textbook originally chosen 
for the class was an introductory text for third grade ESOL students and proved too easy for 
the majority of the class. Based on these responses, we substituted more difficult books free of 
charge within three days. Informal interviews afterward indicated that most parents 
appreciated the prompt response and were satisfied with the new textbook. 

A midterm evaluation survey was distributed to all students except those in the K-2 class two 
weeks into the program. The major purpose of the survey was to examine students' and 
parents' perspectives of the curriculum and to investigate areas for improvement over the 
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remaining two weeks. The questionnaire included 15 5-point Likert scale questions ranging 
from strongly agree (5) to strongly disagree (1), as well as three open-ended questions. The 
midterm evaluation survey results indicated that both parents and students were generally 
satisfied with class activities, assignments, and tests. The mean score of overall satisfaction was 
3.8 out of 5. Further, over 70% students said they enjoyed coming to class. 

Classroom observations were conducted in three classes (grades 5-6, grades 7-8, and grades 
9-10) for approximately 30 minutes in week 3 and were discussed afterward with the teachers 
and students. Observations determined areas that needed assistance and provided a better 
understanding of each class structure. Data from classroom observations also revealed that 
teachers and volunteers collaborated well on the delivery of instruction as well as on 
classroom management to meet the learning needs of their diverse students. Students were 
attentive and well behaved in classrooms. 

Summative evaluation. Summative evaluation data were collected to determine the extent 
to which students made gains in English proficiency and the extent to which the program 
increased students' self-concept. Additionally, satisfaction surveys were administered to all 
participants to seek out suggestions for the next year's program. 

Pre- and posttests developed by classroom teachers were conducted in order to examine 
students' improvements in English proficiency. The pretest took place on the first day of the 
program, while the posttest was conducted two days before graduation. The test results 
showed improvement for all students after completing the program. The t-statistic for related 
samples analysis revealed that the mean differences in all classes were statistically significant 
except in the grades 3-4 class (see Table 2). We also conducted pre- and posttests for the adult 
class, but because the identified design components were not applied to the class, the results 
are not included here. The adult class was offered to mothers whose children were enrolled in 
the program, primarily as a social and conversational practice period while waiting for their 
children, so less regard was given toward improving the quality of this class. However, 
participants did indicate a strong satisfaction with the class and the instructor, who had over 
20 years of experience teaching adult English. 


Table 2 

Summary of English Skills Pre- and Posttest Results 


Class 

N 

Pre-M 

Post-M 

M-diff 

Pre-SD 

Post-SD 

t value 

Grades 3-4 

9 

81.4/100 

92.0/100 

- 10.6 

23.3 

5.9 

-1.70 

Grades 5-6 

13 

62.2/100 

72.2/100 

- 10.0 

15.8 

20.5 

_4 94 *** 

Grades 7-8 

12 

52.5/100 

75.3/100 

- 22.8 

18.1 

11.1 

-5 21*** 

Grades 9-10 

7 

11.6/30 

17.6/30 

- 6.0 

4.8 

5.3 

-5.39** 

TOEFL 

14 

14.7/20 

17.1/20 

-2.4 

4.2 

2.6 

-2.76** 

SAT Reading 


11.1/24 

13.1/24 

- 2.0 

5.6 

4.4 

- 2 . 6 ** 

Essay 

19 

3.8/6 

4.3/6 

-0.5 

1.2 

1.1 

_2 2 ** 

Math 


13.8/20 

14.9/20 

- 1.1 

3.3 

3.5 

-0.95 


N = sample size; Pre-M = mean in the test before the program/possible scores; Post-M = mean in the test after the 
program/possible scores; M-Diff = Pre-M - Post-M; Pre-SD = SD in the test before the program; Post-SD = SD in 
the test after the program; t value= paired sample T test scores; **p < 0.05; ***p < 0.001. 
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A self-concept questionnaire was distributed to measure the overall self-concept of students 
and to examine whether the summer program influenced their self-concepts. Students in 
grades 3-12 were measured at the beginning and end of the program using the Piers-Harris 
Children's Self-Concept Scale, Second Edition (Piers-Harris 2); the questionnaire took 
appropriately 10 minutes. Piers-Harris 2 includes a 60-item self-reporting questionnaire 
relating to six domain scales: Behavioral Adjustment (BEH), Intellectual School Status (INT), 
Physical Appearance and Attributes (PHY), Freedom from Anxiety (FRE), Popularity (POP), 
and Happiness and Satisfaction (HAP). For each item, participants were asked to indicate 
whether a statement applied to them by selecting either yes or no (Piers & Herzberg, 2002). 

The t-statistic for related samples results showed that student self-concept improved after 
the summer program, but the differences were not statistically significant (see Table 3). The 
total scores (TOT) of Piers-Harris 2 represent an individual's general self-concept, with higher 
scores indicating a higher self-concept. According to Piers and Herzberg (2002), if the total 
score is above 60, the student perceives him- or herself very positively. If the total score is less 
than 39, the person seriously doubts his or her self-worth. A total score between 40 and 59 is 
considered average. The average total score of the students in the summer school was 
approximately 50, indicating that they generally perceive themselves positively. 


Table 3 

Summary of Piers-Harris 2 Pre- and Posttest Results 


Categories 

Pre-M 

Post-M 

Mean-diff 

Pre-SD 

Post-SD 

t-value 

TOT 

47.9 

48.8 

-.92 

8.46 

8.59 

-1.59 

BEH 

49.8 

50.8 

-1.0 

7.89 

8.11 

-1.4 

INT 

47.8 

48.9 

-1.3 

9.14 

8.48 

-1.56 

PHY 

44.2 

45.1 

-1.9 

8.72 

8.92 

-3.18** 

FRE 

51.4 

51.4 

-.05 

8.56 

9.11 

-0.06 

POP 

50.7 

50.2 

.40 

8.53 

8.95 

0.52 

HAP 

46.9 

48.2 

-1.0 

7.42 

7.60 

-1.37 


Pre-M = mean in the test before the program; Post-M = mean in the test after the program; M-Diff = Pre-M - 
Post-M; Pre-SD = SD in the test before the program; Post-SD = SD in the test after the program; t value= paired 
sample T test scores; **p < 0.05. 


A program evaluation survey was performed with students and parents at the end of the 
program. The survey included two sets of 4-point Likert scale questions ranging from strongly 
agree (4) to strongly disagree (1), as well as two yes/no questions and five open-ended 
questions focusing on participants' satisfaction with the program. The survey results revealed 
that students' overall reactions to the program were very positive. Over 82% (N = 67) of 
students indicated that they were satisfied with the program. Table 4 summarizes students' 
mean scores for each criterion related to program satisfaction by class level. This table shows 
that students in all classes perceived that the program helped them improve their writing 
scores. Across the class, the mean scores were over 3.1 out of 4.0. 


The Professional Educator 







A Summer School for English Language Learners 


Table 4 

Mean Scores on Program Satisfaction Items by Class (on a scale of 1-4) 


Items 

3-4 

5-6 

7-8 

9-10 

TOFLE 

SAT 

Total 

This summer school helped me 
improve my reading skills 

3.4 

3.2 

2.9 

2.7 

2.5 

3.0 

2.9 

This summer school helped me 
improve my writing skills 

3.6 

3.1 

3.1 

3.0 

3.3 

3.4 

3.2 

This summer school helped me 
improve my communication skills 

3.3 

3.4 

2.8 

2.9 

2.7 

2.5 

2.9 

This summer school helped me feel 

confident in my English skills 

3.3 

3.0 

3.1 

2.7 

2.8 

2.8 

2.9 

Overall, I am satisfied with this 

3.8 

3.1 

2.9 

3.0 

2.8 

3.1 

3.1 


summer school program. 


To identity the activities that helped students improve their English proficiency the most, 
we asked students to indicate their level of agreement on the effectiveness of program 
components. The results (see Table 5) indicated that students perceived the reading and 
writing activity as the most helpful (M = 3.1), followed by class assignments (M = 2.9) and the 
Keypal exchange activity (M = 2.9). Students indicated that both their regular teachers and the 
volunteer teachers were very helpful. The average mean scores of regular teachers and 
volunteer teachers were 3.3 and 3.2, respectively. 


Table 5 

Mean Scores on the Program Effectiveness Items by Class (on a scale of 1-4) 


Items 

3-4 

5-6 

7-8 

9-10 

TOEFL 

SAT 

Total 

The class assignments helped me 
improve my English skills. 

3.4 

2.7 

3.0 

2.6 

2.9 

3.0 

2.9 

The reading and writing competition 

helped me improve my English skills. 

3.1 

2.6 

2.9 

2.7 

2.7 

2.9 

3.1 

The keypal exchange helped me 
improve my English skills. 

3.6 

2.3 

2.7 

2.5 

2.6 

2.2 

2.9 

Class tests (quizzes) helped me 
improve my English skills 

3.1 

2.7 

2.7 

2.7 

2.7 

3.3 

2.8 

My teacher was helpful 

4.0 

3.2 

3.4 

3.4 

2.6 

3.3 

3.3 

Teacher aides were helpful. 

4.0 

3.3 

3.2 

3.6 

3.0 

2.8 

3.2 
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Interestingly, although student agreement on the effectiveness of class tests was 
comparatively low (M = 2.8), the open-ended answers revealed that a large group of students 
wanted greater rigor in class work. One student wrote, "Make the class a little bit harder!" 
Another student said, "More tests and more strict and organized classes." Many students also 
expressed that the class should include more active and fun activities and have longer breaks. 

The parents' survey results indicated that parents were also generally satisfied with this 
program. The overall mean score of satisfaction was 3.0 out of 4.0. Parents particularly 
appreciated the volunteer teachers (M = 3.2) and were happy with the fact that their child 
enjoyed attending (M = 3.2). Interestingly, many parents disagreed with the statement, "A 
four-week program was appropriate" (M = 2.5). The open-ended answers revealed that many 
parents would have preferred a longer session (e.g., two months) and thus thought four weeks 
was not enough. 

More than 64% (N = 52) of students wanted to register for next year's program. The major 
reasons students wanted to attend another session were (a) continuing to improve English 
skills, (b) having fun in class, and (c) meeting new friends. Reasons for not wanting to enroll 
included (a) returning to Korea or going to college the next year, (b) deeming the difficulty 
level inappropriate, and (c) wanting to rest during the summer (i.e., the classes were too 
challenging). 

The teacher and volunteer survey included five open-ended questions regarding positive 
experiences and challenges during the program. The survey was distributed via e-mail at the 
end of the program. The results revealed that overall experiences were also very positive. One 
teacher said, "Grading the posttests felt so rewarding because each student improved." One 
volunteer teacher found the experience especially meaningful, leading her to consider 
attending an ESOL graduate school. High school volunteers also bonded with the young 
students. One teacher said, "Some of the middle school students connected with the high 
school age volunteers. That was great." On the negative side, teachers expressed difficulties 
helping students who lacked English proficiency. One volunteer teacher said, "The levels 
within the classes are very diverse. Some of the students who do not speak much English may 
try to act like they don't understand when they really do; they just don't want to do the work." 
Thus, all teachers suggested dividing classes based on English ability level rather than grade 
level. 


Implications for Program Improvement 

Overall positive results support the continued use of the summer English program for 
ELLs. Although test validity is in question because the academic achievement tests were 
teacher-made and not standardized, posttest grades were significantly higher than pretest 
scores for all but one class. In addition, although there was no statistically significant increase, 
student self-concept was maintained or increased in all areas but the area of "popularity." 
Students, parents, teachers, and volunteer teachers all consistently rated the program as being 
satisfactory and helpful. Such positive results are not unique to this program. Summer 
programs overall are rated as having a positive impact on participating students academically, 
socially, and personally. This holds true for programs designed for underachievers who need 
extra support to combat the summer slide (Borman, Overman, Fairchild, & Kaplan, 2004; 
Laird, Feldman, Borman, & Boulay, 2004; Portz, 2004; Roderick, Jacob, & Bryk, 2004), for 
talented students who need extra support for academic enhancement and social skill 
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development (Rinn, 2006), and for ELLs who need to improve English proficiency (Vanderhaar 
& Munoz, 2005). 

Unfortunately, because our focus was solely on initiating the program (e.g., identifying 
needs, finding funds, developing collaborations with local school districts), we could not 
examine any longitudinal influence of the program on student achievement after they returned 
to the schools. Although we developed the program based on state standards, the connection 
to the school district curricula was lacking because the curriculum was developed by hired 
teachers and the two authors who had limited knowledge in this area. However, the school 
districts have acknowledged the positive effects of the program and have agreed to help 
develop the future curriculum by organizing a working group of 14 teachers from two 
districts. We expect that the newly developed curriculum will help us examine the effects of 
the program more rigorously and investigate longitudinal impacts as we continue to improve 
this project. 

Two years of program implementation have provided some insight into how the program 
can be further developed to benefit students with diverse needs. Evaluation results indicate 
that book reading and writing activities, as well as volunteer teachers helping different levels 
of students, appeared to be effective to the program participants. Additionally, we have 
identified the following three areas that need to be considered fully in order to develop a 
quality summer program for ELLs. 

Incorporating Standardized Testing for ELLs 

Identifying and integrating a well-developed, standardized test designed specifically for 
ELLs is critical. Our program evaluation was comprehensive in that we implemented both 
qualitative (interviews, class observations, and written feedback) and quantitative (teacher- 
made tests, standardized tests, and survey questionnaires) evaluations. These evaluations 
measured changes in cognitive and affective domains of student development as well as the 
perceptions of students, parents, and teachers of the program. Evaluations were also 
conducted using formative and summative designs. Such an approach to program evaluation 
provided rich data for assessing effectiveness and determining areas for further improvement. 

However, the program evaluation did face several shortcomings. In 2008, academic 
achievement was measured with the Basic Achievement Skills Inventory (BASI) for students in 
grades 3 through 9. This measure was deemed inappropriate for the students in the program. 
The test was adequate for measuring English language levels in the areas of vocabulary, 
spelling, language mechanics, and reading comprehension, but it was not an appropriate tool 
for evaluating academic achievement changes over five weeks through a short-term 
intervention. Such feedback led us to change the evaluation method to a teacher-made exam in 
2009. However, because the teacher-made exams had no evidence of validity and reliability, 
two other alternatives may need to be taken into consideration in the future. The first option 
would be to identify a well-developed, standardized test designed for use with ELLs. The 
second option would be to develop curriculum-based tests with assistance from English 
language learner curriculum specialists. As a prerequisite for adopting such exams, curricula 
should be developed based on national and state guidelines and local school policies for 
curriculum development in the related subject area. 

Another challenge was using a self-concept inventory that was not adequately normed for 
the student population we served. Because of this, the validity of the results could not be 
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verified. Further efforts need to be made to identify a test inventory that will be able to 
provide valid results when assessing changes in affective domains for participating students. 

Creating Comfortable Learning Environments 

Developing a learning environment where ELLs can comfortably ask questions and share 
issues with friends and teachers in English is important. In our program, 42% of students 
enrolled in 2009 reported that they had resided in the United States for less than one year. 
Some teachers expressed aggravation and frustration toward the students who spoke to one 
another in Korean. These teachers believed that students should have been practicing their 
English when speaking to one another. For these students, lack of social and academic English 
language skills can serve as a major stressor when adjusting to a new school system. Even 
when language is not an issue for students, it is widely reported that transferring schools 
causes many students to experience academic, social, and emotional challenges (e.g., 
developmental stressors discussed by Alspaugh, 1998; Chung, Elias, & Schneider, 1998; 
Tobbell, 2003). 

Taking all of these findings into consideration, the program must define an approach that 
meets the needs of all parties involved. Helpful information can be gathered from up-to-date, 
evidence-based teaching and learning strategies outlined in English as a Foreign Language 
(EFL)/English as a Second Language (ESL) instruction sets. In reference to students using their 
first language in the classroom, some EFL instructors support it, while others are against it. 
The main argument against the use of first language in the classroom is that the second 
language is best learned when students are fully exposed to the "natural" language learning 
environment with no analysis or translation. While such an immersion approach is obviously 
valid, there are also strong arguments for using the first language. Using a fellow student as a 
resource to translate something that is unclear is highly efficient and immensely valuable for 
learners, preventing time lost due to circuitous explanations that could be better spent on 
language practice (Sue Swift, n.d., http://eltnotebook.blogspot.com/2006/ll/using-ll-in-efl- 
classroom.html). Auerbach (1993) also observes a sociopolitical rationale for the use of the first 
language in ESL classrooms: "Starting with the first language provides a sense of security and 
validates the learners' lived experiences, allowing them to express themselves. The learner is 
then willing to experiment and take risks with English" (p. 19). According to research 
conducted by Schweers, Jr. (1999), 57.9% of students in an English learning class responded 
favorably when their teacher used their first language as a means of relaying instructions. The 
reasons noted for the students' positive responses were that difficult concepts could be 
explained easily, new vocabulary words could be defined clearly, comprehension could be 
checked effortlessly, and comfort levels of students were higher in such an environment. 

Developing Curriculum Content 

Although the summer program affected the enrolled students positively overall, the need 
for a change in curriculum content surfaced. A connection must be made between the summer 
program and the regular school year for maximum benefit. This link can be achieved by 
working closely with curriculum specialists in local school systems when developing 
curricula. 

When selecting curriculum content, we recommend including cultural values and norms in 
general and school policies and regulations in particular. No matter where students live, the 
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purpose of education is for children to acquire knowledge, skills, values, and attitudes that 
they need to become contributing members of society while being successful in life (Ballantine, 
2001). Students who are not familiar with cultural norms of the new society may experience 
confusion and loss of confidence. By organizing curricula around the themes of cultural, 
historical, and societal characteristics of the new society, students will be able to gain new 
knowledge and skills about acceptable behavior in their new environment while 
simultaneously learning English skills. The new knowledge and skills can be applied in 
multiple settings, such as schools, extracurricular activities, leisure, family, and the 
community, as well as inter- and intrapersonal contexts. Most schools and classrooms have 
rules of conduct established for students to follow. Such rules of conduct need to be adopted in 
print format and continue to be implemented in the program so that students become more 
familiar with them. Helping students understand other cultures while maintaining preexisting 
pride in their own culture as a unique ethnic group would help in building a healthy, 
bicultural identity. 


Conclusion 

To help ELLs improve their English proficiency and test-taking skills during the summer, 
we developed and implemented a summer English program over the course of two years. The 
program was created to help the increasing number of incoming international residents adjust 
to their new environment. First-year evaluation data indicated a very positive effect on student 
learning, but interview data with parents and teachers revealed areas for improvement. Thus, 
we reexamined parents' needs and restructured the program in collaboration with teachers in 
local schools. Second-year evaluation data indicated that the summer program had more than 
purely academic influence. The positive, encouraging environment helped students achieve 
confidence as well as academic success. Additionally, incorporating book reading and writing 
activities and having volunteer teachers assist students who lacked English proficiency 
appeared to be effective to program participants. 

The success of the summer program lies in accurately identifying the unique needs of the 
students and parents and providing a high quality program to meet those needs. Curricula 
and teaching that reflects the students' unique cultural upbringing and practical needs will be 
able to engage students in learning and promote their academic, personal, and social 
development through the language acquisition program. We hope that our program 
development and evaluation experiences can provide useful insight to educators who want to 
create or improve a similar program for ELLs. 
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